compressing static assets fails with xstatic-bootswatch 3.3.7.0

[Corey Bryant]

While packaging horizon for ocata milestone 1, we are hitting the following error while compressing static assets:

CommandError: An error occurred during rendering /usr/share/openstack-dashboard/openstack_dashboard/templates/horizon/_scripts.html: '\"../bower_components/respond/dest/respond.min.js\"' isn't accessible via COMPRESS_URL ('/horizon/static/') and can't be compressed
dpkg: error processing package openstack-dashboard (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of autopkgtest-satdep:
 autopkgtest-satdep depends on openstack-dashboard (>= 0~); however:
  Package openstack-dashboard is not configured yet.

dpkg: error processing package autopkgtest-satdep (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of openstack-dashboard-ubuntu-theme:
 openstack-dashboard-ubuntu-theme depends on openstack-dashboard (= 3:11.0.0~b1-0ubuntu1); however:
  Package openstack-dashboard is not configured yet.

Paste of above error:
http://pastebin.ubuntu.com/23513344/

The 3.3.7.0 release of xstatic-bootswatch includes lines such as:
<script src="../bower_components/respond/dest/respond.min.js"></script>

which corresponds to the file in the compression error.

Link to 3.3.7.0 commit of xstatic-bootswatch:
https://github.com/openstack/xstatic-bootswatch/commit/49c0d56698bc14ae5e5c18cc00c3f2e290d2e724

We currently have the following django and compressor versions in zesty:
python-django | 1.8.7-1ubuntu9 | zesty | source, all
python-django-compressor | 2.0-3ubuntu1 | zesty | source, all

which are within the bounds of global-requirements and upper-constraints:
http://paste.ubuntu.com/23517114/

I also attempted using newer versions of django and compressor, with the same results:
http://paste.ubuntu.com/23517443/
http://paste.ubuntu.com/23517845/

[Rob Cresswell (robcresswell-deactivatedaccount)]

I'm unable to recreate this; as far as I can tell, the compressor shouldn't try to resolve a path inside an HTML file.

[Radomir Dopieralski (deshipu)]

Are you running collectstatic before running compress?

[Corey Bryant (corey.bryant)]

I was able to recreate this on a zesty machine from source. Here are the steps I used:

Created a zesty container:

lxc launch ubuntu-daily:zesty
lxc exec useful-redbird /bin/bash

In the container:

sudo apt install git
git clone https://github.com/openstack/horizon.git
cd horizon/
git checkout 11.0.0.0b1
sudo apt update
sudo apt install python-pip
cd -
git clone https://github.com/openstack/requirements.git

cd horizon/
sudo pip install . -c ../requirements/upper-constraints.txt

adduser --system horizon
addgroup --system horizon
usermod -a -G horizon horizon

mkdir /usr/share/openstack-dashboard
mkdir /var/lib/openstack-dashboard
cp manage.py /usr/share/openstack-dashboard/manage.py
cp ./openstack_dashboard/settings.py /usr/share/openstack-dashboard/settings.py

mkdir /etc/openstack-dashboard/
cp ./openstack_dashboard/local/local_settings.py.example /etc/openstack-dashboard/local_settings.py
rsync -avrz ./openstack_dashboard/ /usr/share/openstack-dashboard/openstack_dashboard/

ln -s /usr/share/openstack-dashboard/openstack_dashboard/static /usr/share/openstack-dashboard/static
ln -s /etc/openstack-dashboard/local_settings.py /usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py
ln -s /usr/local/lib/python2.7/dist-packages/horizon/static/horizon/ /usr/share/openstack-dashboard/openstack_dashboard/static/horizon

sudo python /usr/share/openstack-dashboard/manage.py collectstatic --noinput
sudo python /usr/share/openstack-dashboard/manage.py compress --force

output from compress:

root@enhanced-catfish:~/horizon# sudo python /usr/share/openstack-dashboard/manage.py compress --force
Found 'compress' tags in:
        /usr/share/openstack-dashboard/openstack_dashboard/templates/horizon/_conf.html
        /usr/share/openstack-dashboard/openstack_dashboard/templates/_stylesheets.html
        /usr/share/openstack-dashboard/openstack_dashboard/templates/horizon/_scripts.html
Compressing... CommandError: An error occurred during rendering /usr/share/openstack-dashboard/openstack_dashboard/templates/horizon/_scripts.html: '\"../bower_components/respond/dest/respond.min.js\"' is
n't accessible via COMPRESS_URL ('/static/') and can't be compressed

[Richard Jones (r1chardj0n3s)]

I'm not sure what the two static-related symlinks are for:

 ln -s /usr/share/openstack-dashboard/openstack_dashboard/static /usr/share/openstack-dashboard/static
 ln -s /usr/local/lib/python2.7/dist-packages/horizon/static/horizon/ /usr/share/openstack-dashboard/openstack_dashboard/static/horizon

They're not necessary. Still trying to figure out why the bootswatch JS <script> links are causing a problem in the above environment, but not in a dev environment.

[Richard Jones (r1chardj0n3s)]

Also, settings.py shouldn't be in /usr/share/openstack-dashboard/settings.py - it must be in the openstack_dashboard directory (it's part of the "openstack_dashboard application")

[Richard Jones (r1chardj0n3s)]

I've reduced the steps to reproduce down to:

  apt-get update
  apt install git python-pip
  git clone https://github.com/openstack/horizon.git
  git clone https://github.com/openstack/requirements.git
  cd horizon
  git checkout 11.0.0.0b1
  pip install . -c ../requirements/upper-constraints.txt
  mkdir /usr/share/openstack-dashboard
  mkdir /var/lib/openstack-dashboard
  cp manage.py /usr/share/openstack-dashboard/manage.py
  mkdir /etc/openstack-dashboard
  cp ./openstack_dashboard/local/local_settings.py.example /etc/openstack-dashboard/local_settings.py
  rsync -avrz ./openstack_dashboard/ /usr/share/openstack-dashboard/openstack_dashboard/
* ln -s /usr/share/openstack-dashboard/openstack_dashboard/static /usr/share/openstack-dashboard/static
  ln -s /etc/openstack-dashboard/local_settings.py /usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py
* ln -s /usr/local/lib/python2.7/dist-packages/horizon/static/horizon/ /usr/share/openstack-dashboard/openstack_dashboard/static/horizon
  python /usr/share/openstack-dashboard/manage.py collectstatic --noinput
  python /usr/share/openstack-dashboard/manage.py compress --force

Which, you might note, already has the settings.py copy command removed (that file is ignored). When I remove the spurious links (marked * above) the compress works as intended. Those links are not necessary to the correct collection of static files, and in fact confuse the heck out of things.

[Richard Jones (r1chardj0n3s)]

Note that if you perform the above to reproduce, and then remove the symbolic links, you'll also need to forcefully reinstall the horizon package as it is polluted by the symbolic link / collectstatic:

  pip uninstall horizon
  rm -rf /usr/local/lib/python2.7/dist-packages/horizon
  pip install . -c ../requirements/upper-constraints.txt

[Corey Bryant (corey.bryant)]

Richard, thanks for taking the time to look at this. I'll update our packaging accordingly.