[Focal] Policy files are missing

Bug #1872671 reported by Albert Damen on 2020-04-14
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Dashboard (Horizon)
Undecided
Chris MacNaughton
Ubuntu Cloud Archive
Undecided
Chris MacNaughton
Ussuri
Undecided
Chris MacNaughton
horizon (Ubuntu)
Undecided
Chris MacNaughton
Focal
Undecided
Chris MacNaughton

Bug Description

[Impact]
[Test Case]
python3-django-horizon:
  Installed: 3:18.2.1~git2020032709.2c4470272-0ubuntu1

After a fresh install of openstack dashboard on focal, apache2 error.log contains hundreds of error message about missing policy files:

[Tue Apr 14 09:12:34.558183 2020] [wsgi:error] [pid 3062:tid 140253993006848] [remote 10.64.255.1:50364] WARNING openstack_auth.policy No policy rules for service 'identity' in /usr/lib/python3/dist-packages/openstack_dashboard/conf/keystone_policy.json
[Tue Apr 14 09:12:34.559486 2020] [wsgi:error] [pid 3062:tid 140253993006848] [remote 10.64.255.1:50364] WARNING openstack_auth.policy No policy rules for service 'compute' in /usr/lib/python3/dist-packages/openstack_dashboard/conf/nova_policy.json and files under ['/usr/lib/python3/dist-packages/openstack_dashboard/conf/nova_policy.d']
[Tue Apr 14 09:12:34.560622 2020] [wsgi:error] [pid 3062:tid 140253993006848] [remote 10.64.255.1:50364] WARNING openstack_auth.policy No policy rules for service 'volume' in /usr/lib/python3/dist-packages/openstack_dashboard/conf/cinder_policy.json and files under ['/usr/lib/python3/dist-packages/openstack_dashboard/conf/cinder_policy.d']
[Tue Apr 14 09:12:34.561703 2020] [wsgi:error] [pid 3062:tid 140253993006848] [remote 10.64.255.1:50364] WARNING openstack_auth.policy No policy rules for service 'image' in /usr/lib/python3/dist-packages/openstack_dashboard/conf/glance_policy.json
[Tue Apr 14 09:12:34.562703 2020] [wsgi:error] [pid 3062:tid 140253993006848] [remote 10.64.255.1:50364] WARNING openstack_auth.policy No policy rules for service 'network' in /usr/lib/python3/dist-packages/openstack_dashboard/conf/neutron_policy.json

The policy files are indeed missing from the package:

dpkg -L python3-django-horizon | grep json$
/usr/lib/python3/dist-packages/horizon/xstatic/pkg/angular/data/errors.json
/usr/lib/python3/dist-packages/horizon/xstatic/pkg/angular/data/version.json
/usr/lib/python3/dist-packages/horizon-18.2.1.dev1.egg-info/pbr.json

Logging in with a normal user account (without admin role) still shows the admin panel and buttons a normal user cannot use, like identity/users "create user". Trying to use these either doesn't work or throws errors like "Unable to retrieve xxx".

Copying the policy files from the source package solves the problem.

[Regression Potential]
Very low, this patch is just adding necessary policy files back into the package that have already existed in prior releases.

Related branches

Albert Damen (albrt) wrote :

This issue seems to be caused by the regenerated SOURCES.txt file from:

horizon (3:18.0.1~git2020021409.bb959361b-0ubuntu3) focal; urgency=medium

  * d/rules: Force regeneration of SOURCES.txt to ensure that generated
    django{js}.mo files are included in binary packages, resolving issues
    with translations (LP: #1862343).

That change includes .mo files in the package, but at the seem time removed a number of files from the package, see the attached file.

Albert Damen (albrt) wrote :

This is still an issue with the package from focal-proposed: 3:18.3.2-0ubuntu0.20.04.1

summary: - Policy files are missing
+ [Focal] Policy files are missing
Albert Damen (albrt) wrote :

Trying to build the focal proposed package, it seems adding the mo files to SOURCES.txt with sed instead of regenerating SOURCES.txt does the job.

I changed debian/rules:
#rm -f horizon.egg-info/SOURCES.txt
sed -i 's/\(.*\)po$$/\1po\n\1mo/' horizon.egg-info/SOURCES.txt

Building the package with the original sources.txt, regenerating sources.txt and changed sources.txt gives:

Counting non .mo files in python3-django-horizon_18.3.2-0ubuntu0.20.04.1_all.deb
Original SOURCES.txt: 4335
Regenerated SOURCES.txt: 4301
Changed SOURCES.txt: 4335

Counting .mo files
Original SOURCES.txt: 0
Regenerated SOURCES.txt: 96
Changed SOURCES.txt: 96

So changing SOURCES.txt adds the same number of .mo files without losing any non .mo file.

Also it doesn't loose any .json files:
Original SOURCES.txt: 21
Regenerated SOURCES.txt: 3
Changed SOURCES.txt: 21

The resulting package works fine and solves the problems for non-admin users.

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in horizon (Ubuntu):
status: New → Confirmed
Albert Damen (albrt) wrote :

Also happens with bionic/ussuri

apt policy python3-django-horizon
python3-django-horizon:
  Installed: 3:18.3.2-0ubuntu0.20.04.1~cloud0
  Candidate: 3:18.3.2-0ubuntu0.20.04.1~cloud0
  Version table:
 *** 3:18.3.2-0ubuntu0.20.04.1~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/ussuri/main amd64 Packages
        100 /var/lib/dpkg/status

Changed in cloud-archive:
status: New → Confirmed
assignee: nobody → Chris MacNaughton (chris.macnaughton)
Changed in horizon (Ubuntu):
assignee: nobody → Chris MacNaughton (chris.macnaughton)
Changed in cloud-archive:
status: Confirmed → In Progress
Changed in horizon (Ubuntu):
status: Confirmed → In Progress
Changed in horizon:
assignee: nobody → Chris MacNaughton (chris.macnaughton)

Fix proposed to branch: master
Review: https://review.opendev.org/734075

Changed in horizon:
status: New → In Progress

Reviewed: https://review.opendev.org/734075
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=f7dc499de44a575fa2966443784af3d14efcc6ae
Submitter: Zuul
Branch: master

commit f7dc499de44a575fa2966443784af3d14efcc6ae
Author: Chris MacNaughton <email address hidden>
Date: Mon Jun 8 15:06:17 2020 +0200

    Update MANIFEST for dashboard config

    The various policy files in openstack-dashboard/conf are not
    included in the package by default. Update MANIFEST.in to
    recursively include that directory.

    Change-Id: I889a639b5d8d696df997c031036ffb8f88aa66b9
    Closes-Bug: #1872671

Changed in horizon:
status: In Progress → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package horizon - 3:18.3.2-0ubuntu2

---------------
horizon (3:18.3.2-0ubuntu2) groovy; urgency=medium

  * d/p/fix-skipped-config-files.patch: Ensure that config files
    are included in the package (LP: #1872671).

 -- Chris MacNaughton <email address hidden> Fri, 12 Jun 2020 11:20:25 +0100

Changed in horizon (Ubuntu):
status: In Progress → Fix Released

Reviewed: https://review.opendev.org/735310
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=bed596616959e649e6d7215dc136c1f7616cc532
Submitter: Zuul
Branch: stable/ussuri

commit bed596616959e649e6d7215dc136c1f7616cc532
Author: Chris MacNaughton <email address hidden>
Date: Mon Jun 8 15:06:17 2020 +0200

    Update MANIFEST for dashboard config

    The various policy files in openstack-dashboard/conf are not
    included in the package by default. Update MANIFEST.in to
    recursively include that directory.

    Change-Id: I889a639b5d8d696df997c031036ffb8f88aa66b9
    Closes-Bug: #1872671
    (cherry picked from commit f7dc499de44a575fa2966443784af3d14efcc6ae)

tags: added: in-stable-ussuri
Changed in cloud-archive:
status: In Progress → Fix Released
Changed in horizon (Ubuntu Focal):
status: New → Triaged
Changed in horizon (Ubuntu Focal):
assignee: nobody → Chris MacNaughton (chris.macnaughton)
assignee: Chris MacNaughton (chris.macnaughton) → nobody
assignee: nobody → Chris MacNaughton (chris.macnaughton)
description: updated
Changed in cloud-archive:
status: Fix Released → Fix Committed
Corey Bryant (corey.bryant) wrote :

This bug was fixed in the package horizon - 3:18.4.2~git2020070209.392bc2482-0ubuntu1~cloud0
---------------

 horizon (3:18.4.2~git2020070209.392bc2482-0ubuntu1~cloud0) focal-victoria; urgency=medium
 .
   * New upstream release for the Ubuntu Cloud Archive.
 .
 horizon (3:18.4.2~git2020070209.392bc2482-0ubuntu1) groovy; urgency=medium
 .
   * New upstream snapshot for OpenStack Victoria.
   * d/control: Align (Build-)Depends with upstream.
   * d/p/fix-skipped-config-files.patch: Dropped. Fixed upstream.
   * d/control: Update Standards-Version to 4.5.0.
 .
 horizon (3:18.3.2-0ubuntu2) groovy; urgency=medium
 .
   * d/p/fix-skipped-config-files.patch: Ensure that config files
     are included in the package (LP: #1872671).
 .
 horizon (3:18.3.2-0ubuntu1) groovy; urgency=medium
 .
   * New upstream release for OpenStack Ussuri (LP: #1877642).

Changed in cloud-archive:
status: Fix Committed → Fix Released
Corey Bryant (corey.bryant) wrote :

This Chris. Merged and uploaded to focal unapproved queue.

Hello Albert, or anyone else affected,

Accepted horizon into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/horizon/3:18.3.2-0ubuntu0.20.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in horizon (Ubuntu Focal):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-focal
Albert Damen (albrt) wrote :

I created a new horizon host on focal and installed openstack-dashboard.

openstack-dashboard:
  Installed: 3:18.3.2-0ubuntu0.20.04.1

apache error log showed many errors about missing policy files
admin panel was visible while logged in as non-admin user

I enabled focal-proposed and upgraded openstack-dashboard

openstack-dashboard:
  Installed: 3:18.3.2-0ubuntu0.20.04.2

no error messages about missing policy files
admin panel was not visible for non-admin user
started and deleted new instance as non-admin user
switched to admin user and verified admin operations still worked

tags: added: verification-done-focal
removed: verification-needed-focal

The verification of the Stable Release Update for horizon has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package horizon - 3:18.3.2-0ubuntu0.20.04.2

---------------
horizon (3:18.3.2-0ubuntu0.20.04.2) focal; urgency=medium

  * d/p/fix-skipped-config-files.patch: Ensure that policy files are
    included in the package (LP: #1872671).

 -- Chris MacNaughton <email address hidden> Wed, 01 Jul 2020 14:54:51 +0000

Changed in horizon (Ubuntu Focal):
status: Fix Committed → Fix Released

Hello Albert, or anyone else affected,

Accepted horizon into ussuri-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:ussuri-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-ussuri-needed to verification-ussuri-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-ussuri-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-ussuri-needed
Albert Damen (albrt) wrote :

I created a new horizon host on bionic with bionic/ussuri and installed openstack-dashboard.

openstack-dashboard:
  Installed: 3:18.3.2-0ubuntu0.20.04.1~cloud0

apache error log showed many errors about missing policy files
admin panel was visible while logged in as non-admin user

I enabled bionic/ussuri-proposed and upgraded openstack-dashboard

openstack-dashboard:
  Installed: 3:18.3.2-0ubuntu0.20.04.2~cloud0

no error messages about missing policy files
admin panel was not visible for non-admin user
started and deleted new instance as non-admin user
switched to admin user and verified admin operations still worked

tags: added: verification-ussuri-done
removed: verification-ussuri-needed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers